YET-1269: /manage-mac skill — CLI-first MaC YAML authoring#88
Merged
Conversation
- Add schemas/mac-schema.json (generated from monolith-django) - Add /manage-mac skill for schema-aware MaC YAML authoring (create, edit, validate) - Add plugin symlinks in all 5 editor plugins - Inject schema validation gate into monitoring-advisor data-monitor-creation.md - Inject schema validation rules into tune-monitor SKILL.md - Update README.md and skills/README.md to list manage-mac Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
wojtekdeka
changed the title
…ope disclaimer - Regenerate mac-schema.json: deprecated field aliases (resource, comparisons, notify_rule_run_failure, domain, domain_uuids) marked with deprecated: true, root additionalProperties removed for dbt YAML co-location compatibility - Add schema scope disclaimer to manage-mac, monitoring-advisor, and tune-monitor skills: schema validates structural correctness only, not cross-field semantics Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
wojtekdeka
changed the title
…ated field migration guidance - Rebundle mac-schema.json: fields with deprecated aliases no longer in required, canonical fields carry description pointing to old name for bidirectional agent lookup - manage-mac: add deprecated field migration workflow — agent proactively lists all deprecated fields found and offers to migrate after explicit user confirmation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…specific authoring guidance - Add import workflow using get_monitors(config_format="yaml") - Fix validate check #5: allow extra top-level keys for dbt co-location - Add inline validation.alert_condition predicate tree example (schema gives no structure) - Add metric.data_source structure note (no flat table field) - Add auto threshold callout: operator AUTO/AUTO_HIGH/AUTO_LOW, alert_conditions required - Add sensitivity lowercase enum callout - Regenerate mac-schema.json with description hoisting and notifications: key - Add MaC deployment fork to monitoring-advisor data-monitor-creation.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Scope sensitivity field to metric monitors only (not valid on custom_sql) - Add audiences vs notifications: NaC distinction to Phase 0 - Add description to metric required fields list in Phase 0 - Expand validation template to include all required fields (data_source, schedule) - Note alert_condition singular vs alert_conditions plural on validation - Fix validate example: full_table_id → description (field that actually exists); move sensitivity typo example to metric type; add custom_sql sensitivity as wrong-type example - Edit workflow: Add instruction directs to Create workflow field-gathering steps; Remove instruction handles empty list by removing the type key Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- P0: Add explicit notes that query_performance and table monitor type do not have a flat table: field (use selection/asset_selection instead) - P1: Scope AUTO/AUTO_HIGH/AUTO_LOW operator callout to metric monitors only; note custom_sql and field_quality use GT/LT/GTE/LTE/EQ/NEQ - P1: Wire deprecated field detection into Edit workflow Step 2 - P2: Update ambiguous-intent fallback question to surface Import workflow - P2: Make Import Step 3 namespace prompt conditional (mirrors Create Step 3) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…cs, add import triggers - Expand ambiguous-intent fallback to list all 4 workflows explicitly (Create / Edit / Validate / Import) - Document binary predicate right-side value node requirement for starts_with, regex_match, etc. - Add Import example triggers to when_to_use frontmatter Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ontmatter Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…iptions, add tune-monitor awareness - Remove bundled schemas/mac-schema.json; all skills now fetch from https://docs.getmontecarlo.com/mac/schema.json via WebFetch — no manual sync required - Update manage-mac SKILL.md: Phase 0 uses WebFetch, frontmatter description reflects all 4 workflows (create/edit/validate/import), graceful degradation updated - Update tune-monitor Phase 5: schema validation uses published URL; new step warns that API changes to MaC-managed monitors will be overwritten on next apply and offers /manage-mac edit workflow as the correct path - Update data-monitor-creation.md schema reference to published URL - Update all 5 plugin CHANGELOGs: consistent /manage-mac description (create/edit/validate/import), remove stale "bundled locally" line Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t/validate/import Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|
||
| The top-level structure under `montecarlo:` contains these monitor type keys: | ||
|
|
||
| - `field_health`, `dimension_tracking`, `json_schema`, `metric`, `metric_comparison` |
Contributor
There was a problem hiding this comment.
field_health/dimension_tracking/field_quality/comparison are deprecated - maybe we need to properly annotate this on the json schema to generate this skill.
fresheness/volume are discouraged and I would remove them from the skill schema if possible as well.
Contributor
Author
There was a problem hiding this comment.
The rewritten skill actively declines all deprecated types and suggests the nearest valid replacement. Annotating them in the schema itself is a good follow-up — filed separately.
| confuse them. | ||
|
|
||
| `validation` monitors have an `alert_condition` field (singular — not `alert_conditions`) | ||
| whose internal structure is a predicate tree not described further by the schema. The minimal |
Contributor
There was a problem hiding this comment.
The predicates definitions can be fetched with the get predicates MCP tool.
Contributor
Author
There was a problem hiding this comment.
Done — skill calls get_validation_predicates in Create Step 2 before authoring a validation monitor, and stops if the result is empty.
Comment on lines
+225
to
+228
| While reading the file, also check for deprecated field names (marked `deprecated: true` in | ||
| the schema). If found, follow the deprecated field handling procedure in the Validate section | ||
| before applying changes. | ||
|
|
Contributor
There was a problem hiding this comment.
Maybe we can have a separate published file without deprecated fields... to avoid confusing the LLM with constructs we won't longer accept.
Comment on lines
+3
to
+12
| description: Create, edit, validate, and import Monitors-as-Code YAML files. Fetches the published MaC JSON Schema to ensure correctness across all 14 monitor types. | ||
| when_to_use: | | ||
| Invoke when the user has a MaC YAML file they want to create, edit, or validate, or when they | ||
| want to export live monitors into a MaC YAML file. | ||
| Example triggers: "create a monitors YAML for this table", "add a metric monitor to my MaC file", | ||
| "validate my monitors.yaml before I apply it", "what's wrong with my MaC file", | ||
| "export my existing monitors to YAML", "get my monitors into a file so I can commit them", | ||
| "import my live monitors to YAML", "get a MaC file from my existing monitors". | ||
| Do NOT invoke when the user wants to discover what to monitor or generate monitors from scratch | ||
| via table exploration — use monitoring-advisor for that. |
Contributor
There was a problem hiding this comment.
For monitor creation, a ton of knowledge is part of the data-monitor-creation skill. Can we leverage that somehow?
Contributor
Author
There was a problem hiding this comment.
Done — Create Step 2 references ../monitoring-advisor/references/data-monitor-creation.md for table/field/UUID resolution steps.
- Remove deprecated/discouraged monitor types from supported type list: field_health, dimension_tracking, field_quality, comparison, freshness, volume - Keep single "do not author" note; removed all scattered mentions - Strengthen get_validation_predicates usage: always call the tool, list is reference only - Remove field_quality and volume from AUTO operator callout — verify via schema instead - Create Step 1: reference data-monitor-creation.md for table/field/domain/warehouse validation instead of duplicating that knowledge Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Create and Edit workflows now call create_or_update_*_monitor(dry_run=True) to get canonical YAML from the backend rather than hand-authoring from the JSON schema. The schema (WebFetch) is now only used in the Validate workflow for local validation without a server round-trip. Key changes: - Remove mandatory Phase 0 schema fetch; schema is now Validate-only - Add MCP tools table to Prerequisites; MCP server required for Create/Edit/Import - Create: tool-first via dry_run=True, with table/column validation from data-monitor-creation.md steps 1-3 - Edit: dry_run=True for both deployed (UUID via get_monitors) and undeployed monitors - Validate: unchanged — WebFetch schema, graceful degradation, type-specific reminders - Import: unchanged — get_monitors with config_format=yaml - Fallback to schema-based authoring for types without a dedicated MCP tool (json_schema, query_performance, bulk_monitor) - Version bump 1.0.0 → 2.0.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Create Step 1: prompt for metric type/expression on `metric` monitors - Validate: document `query_performance` alert_conditions structure (require `threshold` + `metric`; additionalProperties: false) - Validate: document `bulk_monitor.monitor_type` enum values (`bulk_metric`, `bulk_pii`); `metric` is not valid Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
santiagoaguiar
approved these changes
May 21, 2026
Comment on lines
+61
to
+67
| For monitor types without a dedicated MCP tool (`json_schema`, `query_performance`, | ||
| `bulk_monitor`), fall back to schema-based authoring: fetch the schema via Bash and derive all | ||
| fields from it. Never guess field names. | ||
|
|
||
| ```bash | ||
| curl -s https://clidocs.getmontecarlo.com/mac/schema.json | ||
| ``` |
Contributor
There was a problem hiding this comment.
Make sense, we should eventually add tools for these as well 👍 .
| `comparison`, `freshness`, or `volume`. If the user explicitly requests one of these, | ||
| decline: inform them it is no longer supported, and suggest the closest valid alternative | ||
| (e.g. `freshness` or `volume` → `metric` monitor tracking recency or row count; | ||
| `field_health` → `validation` monitor; `comparison` → `metric_comparison`). |
Contributor
There was a problem hiding this comment.
Suggested change
| `field_health` → `validation` monitor; `comparison` → `metric_comparison`). | |
| `field_quality` → `validation` monitor; `comparison` → `metric_comparison`; `field_health` → `metric`). |
Comment on lines
+118
to
+120
| After receiving the dry_run output, check it for deprecated field names (e.g. `resource`, | ||
| `notify_rule_run_failure`, `domain_uuids`, `comparisons`). If any are present, replace them with | ||
| their canonical equivalents before assembling the file. |
Contributor
There was a problem hiding this comment.
Should never happen, export should use 'modern' names.
Contributor
Author
There was a problem hiding this comment.
Good point, removed — backend already returns modern names.
Comment on lines
+165
to
+172
| - If the monitor has been deployed and has a UUID: if the UUID is already known (e.g. present | ||
| in the file or supplied by the user), use it directly. Otherwise resolve it via | ||
| `get_monitors(full_table_id=..., config_format="yaml")`. Then call | ||
| `create_or_update_*_monitor(dry_run=True, monitor_uuid=<uuid>, ...)` with the updated | ||
| parameters. Use the returned YAML block to replace the existing monitor entry. | ||
| - If the monitor has not yet been deployed (no UUID): call | ||
| `create_or_update_*_monitor(dry_run=True, ...)` without a UUID, using the current parameters | ||
| plus the requested changes. Replace the existing block with the returned YAML. |
Contributor
There was a problem hiding this comment.
Given here we are strictly in the YAML realm, we shouldn't need to resolve the UUID, we only need to make sure that for edit we preserve the name attribute and use the same namespace, otherwise it will be a different monitor. You can still use create_or_update with dry_run=True to get the shape of the YAML. Pass the current monitor name as the name argument to create_or_update_*_monitor.
Contributor
Author
There was a problem hiding this comment.
Switched to name-based identity throughout — no UUID lookup needed.
Comment on lines
+194
to
+198
| Some fields are YAML-level only and do not require a `dry_run` MCP call: `audiences`, | ||
| `is_paused`, `labels`, `tags`, `priority`, `data_quality_dimension`, `domains`. For these, | ||
| add or modify the field directly in the YAML without calling the MCP tool. Note: `is_paused` | ||
| is accepted by the backend at apply time but is not present in the JSON Schema — if a schema | ||
| scan flags it as unknown, ignore that warning. |
Contributor
There was a problem hiding this comment.
I think these should be present already in create_or_update_*_monitor. Maybe instead of mentioning specific properties here provide a more general statement without actual examples to reduce drift.
Contributor
Author
There was a problem hiding this comment.
Done, replaced with a general statement.
Comment on lines
+200
to
+209
| ### Step 3: Apply and show the diff | ||
|
|
||
| Show only what changed — before/after for modified blocks, or the new block for additions. Then | ||
| write the updated file using the Edit tool. | ||
|
|
||
| Ensure the `# yaml-language-server: $schema=https://clidocs.getmontecarlo.com/mac/schema.json` | ||
| header is the first line. Add it if the existing file lacks it. | ||
|
|
||
| If removing the last monitor of the last type, the file should contain only the | ||
| yaml-language-server header and `montecarlo: {}`. |
Contributor
There was a problem hiding this comment.
Should we add the required CLI call here?
Contributor
Author
There was a problem hiding this comment.
Yes, added compile + apply to Edit Step 3.
Comment on lines
+213
to
+215
| ## Validate workflow | ||
|
|
||
| ### Step 1: Fetch the schema |
Contributor
There was a problem hiding this comment.
Should we instead add support to the CLI to run a local validation, then here instruct to use CLI?
Contributor
Author
There was a problem hiding this comment.
Yes! Rewrote Validate to use montecarlo monitors compile as Step 1 — manual fallback only when CLI is unavailable.
Comment on lines
+317
to
+319
| ## Import workflow | ||
|
|
||
| ### Step 1: Identify the source |
Contributor
There was a problem hiding this comment.
The CLI has support to convert from UI to MaC and viceversa, and to export monitors to YAML. This procedure might still be more flexible but worth considering a reference to CLI tools.
Contributor
Author
There was a problem hiding this comment.
Added CLI-first path to Import — export and convert-to-mac as Tier 1, get_monitors MCP as fallback.
Comment on lines
+41
to
+45
| ## Prerequisites | ||
|
|
||
| The Monte Carlo MCP server is required for **Create**, **Edit**, and **Import** workflows. No MCP | ||
| tools are needed for **Validate** (file + schema only). | ||
|
|
Contributor
There was a problem hiding this comment.
Should we suggest installing MC CLI for these flows? Is fundamental for apply and others.
Contributor
Author
There was a problem hiding this comment.
Added a Prerequisites section + CLI check at workflow start with install/configure steps and a 3-tier system throughout the skill.
- Fix deprecated type alternatives mapping (field_health → metric, field_quality → validation per Santi's suggestion) - Remove post-dry_run deprecated field check (backend returns modern names) - Edit workflow: use name-based identity, drop UUID resolution - YAML-level fields: replace specific list with general statement - Add CLI-first tooling tier system (Tier 1: CLI, Tier 2: MCP, Tier 3: Manual) - CLI check at workflow start with install/configure instructions - Add Prerequisites section with docs links for CLI and MCP server - Validate workflow: montecarlo monitors compile as primary path - Edit workflow: compile + apply commands in Step 3 - Import workflow: convert-to-mac / export as CLI-first path - Rename tier 3 from "Schema" to "Manual" (customer-facing language) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Namespace enforcement: prompt before showing apply command (S33) - Type routing: map common phrases to monitor types (S39) - Deprecated migration decline path: proceed with edit without migrating (S53) - Multi-monitor: complete all dry_run calls before assembling file (S35) - CLI setup: resume workflow after install/configure (S4) - Monitor removal: explicitly preserve siblings in type list (S48) - YAML-level fields: add common examples (is_paused, labels, priority, etc.) (S56-58) - Validate Tier 3 label: explicit in step heading (S63) - Validate deprecated listing: all occurrences not just unique names (S79) - Entry point: in-body redirect to monitoring-advisor for discovery requests (S93) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
wojtekdeka
changed the title
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the
/manage-macskill for Monitors-as-Code YAML authoring — create, edit, validate, and import MaC YAML files from any MCP-compatible editor (Claude Code, Cursor, etc.).CLI-first, 3-tier approach:
montecarlobinarycompile), apply, import (convert-to-mac,export)dry_run=True, resolve table metadataThe skill checks CLI availability at startup and prompts the user to install/configure if missing, then gracefully falls back through tiers.
Four workflows:
dry_run=True, compile + applymontecarlo monitors compilefirst; manual schema validation as fallbackmontecarlo monitors export/convert-to-macfirst;get_monitorsMCP as fallbackDeprecated type handling: skill actively declines
field_health,dimension_tracking,field_quality,comparison,freshness,volumeand suggests the correct modern alternative.Schema validation gates in
monitoring-advisorandtune-monitor:clidocs.getmontecarlo.com/mac/schema.json) before presenting it to the usertune-monitoralso detects MaC-managed monitors and offers to hand off to/manage-macto keep the YAML file as the source of truthFollow-up: YET-1310 — publish a clean schema without deprecated fields.
Test plan
freshness) — skill declines and suggestsmetricconvert-to-macCLI pathschema.yml— extra top-level keys not flagged/manage-mac🤖 Generated with Claude Code